Enterprises and other organizations typically have computer security infrastructure installed on their networks, such as firewalls, network intrusion detection systems (IDS), network intrusion prevention systems (NIPS), etc. These and other systems collect and log large amounts of data concerning network traffic and events occurring on the various types of computing and networking systems of the organization. Often, such information is significant to the organization's security configuration, indicating factors such as vulnerabilities, contexts, effective and ineffective configurations, successful and blocked attacks, etc. Collected data concerning networking and computing events of importance to the computer security of an organization are referred to herein as “security telemetry.” Security telemetry can be in the form of monitoring data from firewalls, IDS/IPS, vulnerability scanners and the like, log data from network devices (e.g., routers, switches, etc.), servers, endpoints and other computing devices, scanning or other assessment data from cloud, server or endpoint based security systems, etc. Analysis of and intelligence concerning collected security data is also a form of security telemetry.
Many organizations collect such large amounts of security telemetry automatically that it becomes impracticable to analyze or process by the organization, or by using conventional database or enterprise level security tools. It is thus frequently desirable to utilize an outside cloud based security telemetry analytics facilitator, with the ability to process petabytes of data, perform meaningful analysis of these large amounts of varied data, and provide organizations with the ability to query their large volumes of security telemetry in useful ways. Moreover, cloud based analytics facilitators process and store security telemetry for multiple enterprises, which creates concerns about exposing confidential information between parties when making the security telemetry available to its owners and other authorized entities such as third party analytics providers.
Securely storing and provisioning large amounts of security telemetry provided by multiple independent entities creates a number of challenges that it would be desirable to address.